^ 



(N 



X 



Thermodynamics of Markov Processes with 
Non-extensive Entropy and Free Energy 



O . Hong Qian 

(N 

o ! Department of Applied Mathematics, University of Washington 
,^- Seattle, WA 98195 

CN ; February 3, 2011 

C^' Abstract 



Parallel to the recently presented complete thermodynamic formalism for master equation 



C^ . systems IT], we show that a "thermodynamic" theory can also be developed based on Tsallis' 



generalized entropy Sq — [1 — J2i=iPi]/{Q ~ 1) ^^'^ Shiino's generalized free energy F^ 



q 



[^j^-^ Pi{pi/TTi)'^~^ — l]/{q—l) where tt^ is the stationary distribution of the master equation. 
*vj ! dFq/dt = —fd < and it is zero iff the system is in its stationary state. dSq/dt = fd — 

^ ' Qex where Q^x characterizes the heat exchange. For systems approaching equilibrium with 

detailed balance, fd is the product of Onsager's thermodynamic flux and force. However, it 
'^ ' is discovered that the Onsager's force is non-local. This is a consequence of the particular 

transformation invariance for zero energy of Tsallis' statistics. 
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Q ; 1 Introduction 
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Statistical thermodynamics of small systems require the traditional thermodynamic quantities 
such as energy and entropy to be non-extensive |2|. Recently, a complete statistical ther- 
modynamics based on both the Gibbs entropy and Gibbs free energy (relative entropy) has 
been presented for stochastic dynamics based on master equations |1|. The theory intro- 
C^ I duces several key quantities: entropy production rate, free energy dissipation rate, excess and 

house-keeping heat. It is therefore natural to ask whether the thermodynamic formalism can 
be developed in parallel for non-extensive entropy and free energy. This note is to show that 
a similar structure does exist. 

Generalized Entropy. The mathematics of entropy is intimately related to the logarith- 
mic function, y = \n{x), which is the inverse of exponential function x = exp(j/). The 
exponential function is defined through the limit: 

exp(j/) = lim (1 + ry)^ . (1) 

■I — >o 

The inverse function of the right-hand- side of Eq. ([T]l is 



r"" - 1 

y = - -■ (2) 



If one takes this as the analogue of the y — ln(a;), then corresponding to Gibbs' entropy 
X^i Pi In Pi, one can have a generaUzation of the entropy: 

i=i ^ ^ 

If we denote r = q — 1, then the Eq. ^ becomes 

g = l^^=l^'^ _ (4) 
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This is the non-extensive entropy first proposed by Tsallis |l3|. Because of relations in Eqs. 
([TJ and (|2]i, the Tsallis entropy enjoys many of the properties of the Gibbs' entropy. 

Relative entropy. Recognizing the correspondence between ln(a;) and [x'^^^ — l)/(g — 
1), a generalization of the relative entropy with respect to a distribution g, can be similarly 
introduced f4l: 

H,{{pM{9r}) = r^ ( > >. ( - r ' - 1 1 • (5) 
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Statistical internal energy and Hq as the Gibbs free energy. Assuming {TTi} is a station- 
ary distribution to the master equation in Eq. ( fT2] l below, one can define a statistical internal 
energy by applying Boltzmann's law in reverse UJ: 

TTJ'" - 1 

e^ = ^ — , (6) 
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in analogous to energy — ln(l/7ri). Then the mean statistical internal energy is 

The mean internal energy being the sum of pfei rather than the conventional p^Ci is a peculiar 
feature of the Tsallis statistics. Then we have generalized free energy: 

F,^U,-S, = —^{y]p,[^f -l|=H({p,}||{7r,}). (8) 



^S''(s 



Conditional entropy. Tsallis entropy does not enjoy the standard equation concerning 
conditional entropy of a pair of correlated random variables A and B IS): 

S{AB)=S{A) + Sa{B), (9a) 

where 

SiAB) = ^K<Zylog(Kg.y), (9b) 

SiA) = J2p,\ogp,, (9c) 

i 
Sa{B) = J2 p A J^q^l^Og q,A . (9d) 



For Tsallis's entropy with q > 1, one has 

S,{AB) - 






< S{A)+Y,P^S{B\A = ^) 

i 

Eq. (|9]l, together with entropy reaches its maximum with uniform distribution 

5'({pi,P2,P3,--- ,Pn}) < S {<-,-,-,■■■ ,-\] , (10) 

\in n n "J/ 

and entropy is invariant with adding an impossible event 

S{{P1,P2,P3,--- ,Pn,0}) = S({pi,P2,P3,--- ,Pn}), (H) 

provide Khinchin's uniqueness theorem for Gibbs entropy fSl. TsalHa entropy does satisfy 
both Eqs. (QOl and (dTJ. 

2 Generalized Thermodynamics of Markovian Processes 

Relative entropy and positive free energy dissipation. We now consider a Markov process 
with master equation 

^^ = 2^ Pj Wj^ - PiWij . (12) 

We assume both {pi{t)} and {^^(t)} satisfy the same master equation, with different initial 



distributions. Then the time derivative of the relative entropy 



-rHJt) = > 

dt ^^ ' q-1^ 

Q-l /^ 



"^gdt)) dt ^'^ '\9^it)J dt 



-('?-l).9j 
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q 
< 0. 



^y^ !^i (P^9l 

-Kjt.gV I V.9.P. 



\9iPj 



(13) 



The last step is because Wij > when i ^ j, the summand is zero when i — j, and the term 
in the {•••}: 



qz 
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g — 1 (iz\l — z 



<0, (z>0). 



(14) 



Now if we take a stationary distribution {iTi} as the {17^}, then we have the time derivative 
of generalized free energy Fg-. 



dF, 
dt 
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where 



(f>ij (i) = PiW^j - PjWji , $jj (t) 





(15) 




(16) 


{p^/nif~^ - {pj/lTj) 
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Markovian dynamics with detailed balance. Master equation with detailed balance is a 
special class of stochastic systems which represent closed systems approaching to equilib- 
rium. For master equation with detailed balance, iiiWij = T^jWji is equivalent to the Kol- 
mogorov cycle condition; 

(l5) 



^10*1 "^Zl?2 



i-lin ^^inlQ 



1, 



for any cycle path io,ii, 



,i„_i,i„,zo M- Then we have 

2 \ (PiW; 



i^jWjr)"'^ + (^2Wy)«-l 



>9-l 



tj ) 



(PjWji) 
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(19) 



In this case, the (pij and <l>y can be identified as Onsager's thermodynamic flux and force, 
respectively. Therefore, we shall call their product /ree energy dissipation rate 



dF, 
~di 



N 



»J=1 



(20) 



For standard extensive thermodynamics, the Onsager's thermodynamic force $y is "lo- 



cally determined". It depends only on pi, pj, 



and Wji. The local energy difference 



a — Cj = \n{wij/wji). However, Eq. (fT9] l shows that the $ij here dependens also on station- 
ary TTi and TTj, not just their ratio which can be determined locally (Or equivalently, it depends 
on the absolute value of e^ and ej, not just their difference.) This is a consequence of the 
particular transformation invariance for zero energy of Tsallis' statistics. 

The balance equation for the Tsallis entropy. With or without detailed balance, we have 



dSS) 



N 



dt 






'}rJ"\ '-' 
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(21) 
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where statistical internal energy ei is defined in Eq. (|6]l from the stationary distribution tt^ 
The last term in Eq. (ISTT l can be identified as excess heat exchange rate Q: 



N 
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(22) 



The Qea; = in the steady state. However, for equiUbrium steady state with detailed 
balance, each term in the summand is zero. However, for nonequilibrium steady state, the 
summand is not zero. But the sum is zero due to Tellegen theorem: The "heat, or kinetic 
energy" continuously circulates in the system in nonequilibrium steady state. 

Entropy production rate in master equation systems with microscopic reversibility. So 
far, the Markov processes we have considered are quite general. Now we consider master 
equations which have Wij ^ iff Wji ^ 0. For such systems, time reversal trajectory can 
be defined and studied. It has been show that the sufficient and necessary condition of a 
stationary process to be time reversible is the entropy propduction rate being zero [6J. We 
shall now generaUze this result to the non-extensive setting. 

It turns out that the excess heat can be further decomposed into house keeping heat and 
heat dissipation rate 

Qex = Qhk — hd (23) 

where 



ihk = 



2{q 



N 



ttS 



i.i=l 



2(9-1) 



ly' {{n.,w.,Y-' - 1) - (^y^' {^,^,if-' - 1) 



g-1 ( fl-l 



1 



p] (w;J^ -1 



(24) 
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The house keeping heat Qhk \1\ characterizes the amount of work done from outside to keep 
a system away from equilibrium. It is zero in equilibrium steady state with detailed balance 



T^iWij — T^jWji yi,j. In fact Qhk ^ always; 
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Ihk 



N 



{PiWi 



I ^^ij 



i,j = i 
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(PjWjj) 
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^ 9YI iP^'Wij - Pj'Wji) 
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{P^W^J^ - (PjWji) 
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ij = l 
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X 9-1' 



> 0. 



The first term is greater than zero since if piWjj > pjWjj,then ((piWij)'"^ — (pjWji)'^~^'j /{q~ 
1) > 0. It is known as the entropy production rate tp. The second term is in fact the fd which 
is also greater than zero. More importantly is Cp — fd~ Qhk > 0. 

Therefore, the total entropy production rate, Cp has two non-negative contributing terms 
fd and Qhk which represent two different origins of irreversibility: system's transient spon- 
taneous relaxation toward its steady state and environment's sustained driving away from 
equilibrium. For closed system, Qkh = for all time. 
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